What can ChatGPT do? 

One of the noteworthy and powerful features of ChatGPT is its ability to generate text that is difficult to distinguish from a human-generated text. This feature has the potential to provide numerous possibilities for generating automatic text, summaries, paragraphs, essays, articles, and other types of writing without human intervention. Examples of what ChatGPT can do are listed below in order to provide an overview of the variety and versatility of its functionality in generating text using simple prompts:

• Give definitions, explanations, ideas, etc.
• Write an essay based a topic prompt with a set number of words, including citations and references, and spelling style (British or American)
• Write introductions, abstracts, stories, and poems 
• Create a detailed course description and outline along with specific topics
• Create an assignment for a course and provide specific instructions based on given rubrics and percentages 
• Offer suggestions for feedback on student essays and assignments
• Provide feedback on a written text and ways in which it can be improved 
• Generate computer codes based on a set of instructions (e.g. python code, STATA: data science statistical software application code)
• Write emails for different purposes  
• Write sport game summaries, and opinion pieces with prompts to advocate for a particular stance or argument on a topic  
• Explain concepts with a rap 

These are just some key examples of the textual artifacts that ChatGPT can generate with very simple question prompts. Of course, ChatGPT has limitations in terms of what it can generate. The creativity of ChatGPT in writing depends, to a large extent, on how one can be creative in asking questions and creating prompts. One well-known limitation of ChatGPT is its uni-modal input capabilities (Susnjak, 2022). In other words, ChatGPT does not accept images, graphs, or videos. This limitation can be creatively used to inform the design of alternative assessment methods and modalities that make use of multimedia artifacts. Furthermore, ChatGPT does not answer political or medical questions and avoids answering opinion and controversial questions (Shah, 2022). 

ChatGPT: Key Concerns 

The introduction of ChatGPT has raised many academic integrity issues and concerns for educational and academic institutions. Many universities, educational institutions, and researchers are in the process of rethinking and reimagining how assessments, exams, and tests can be made plagiarism-proof as ChatGPT can easily be used to generate text for all kinds of assignments, essays, and projects, resulting in potential academic dishonesty. Some of the key concerns are related to the ways in which ChapGPT can be used to potentially violate academic, research, and scholarship integrity, such as writing essays, term projects, and scholarly journal articles. Furthermore, a number of academics have expressed concerns about the possible negative impact of AI tools such as ChatGPT on students’ critical thinking, learning experiences, research training, and imagination. Another area of concern for higher education institutions is the ways in which AI tools such as ChatGPT may have an impact on and implications for student application and admission processes. More specifically, how do we ensure the academic integrity of the applicant’s documentation for admissions such as a statement of purpose and intent or essay requirements for undergraduate and graduate programs?

A recent study of 50 abstracts from 5 high-impact factor journals asked ChatGPT to generate research abstracts based on their titles and journals. The study found that ChatGPT could write believable scientific abstracts with completely generated data. The authors recommend that “abstract evaluation for journals and medical conferences must adapt policy and practice to maintain rigorous scientific standards; they suggest the inclusion of AI output detectors in the editorial process and clear disclosure if these technologies are used” (Gao et al., 2022). The authors also note that the boundaries around what is acceptable for the use of AI-generated scientific papers is yet to be determined. A prominent and empirically tested example of growing concerns surrounding the use of ChatGPT is related to the tests conducted by professors at two universities in the US who carried out experiments with ChatGPT and found that the tool was able to pass law exams in four courses and another exam in business.

This question of whether or not ChatGPT should be allowed in schools and universities is also at the heart of many discussions in the media, K-12, and higher education institutions. There are competing perspectives as to whether or not we should consider using ChatGPT in our teaching and learning. Some institutions and educators argue that ChatGPT should be banned in schools and universities. For instance, the New York education department announced that they had forbidden ChatGPT on all of their networks and devices in public schools (Yang, 2023). The University of Sydney’s latest academic integrity policy specifically states that “generating content using artificial intelligence” as a form of cheating (Cassidy, 2023). In the UK, lecturers have been encouraged to review the ways in which they assess students in their courses, considering that some students are already using AI tools that are capable of creating high-quality essays (Weale, 2023). 

In contrast, some universities in Canada encourage educators and instructors to engage with students and have a transparent conversation about the tool and its limitations as well as how to avoid plagiarism and academic integrity violations when using AI-enabled tools. Academic integrity expert Sarah Elaine Eaton argues that rather than preventing AI-enabled tools such as ChatGPT, we need to critically evaluate how we conduct assessments (Friesen, 2023). This approach would call for an open conversation with students about their learning and how these tools may help or hinder their learning opportunities and experiences. Methods such as take-home exams, out-of-class writing assignments, and online exams are being reevaluated and replaced by non-textual, skills-based, and, in some cases, multimodal tests, exams, and assignments. Examples of the ways in which assignments could be ChatGPT-proof include creating video presentations, podcasts, posters, and in-class artifact and research creations. 

The use of ChatGPT has also major implications for scientific writing and publishing and scholarly communication. Publishers of scientific journals are currently focusing their attention on taking concrete actions and developing author guidelines and policies to ensure that the manuscripts submitted for publication transparently state the authorship or co-authorship status of their manuscripts. For instance, the journal Nature has developed principles that explicitly address the use of ChatGPT as follows: “Large Language Models (LLMs), such as ChatGPT, do not currently satisfy our authorship criteria.” It further stresses that the authors are required to explicitly state how they have used the tool in their methodology section (Nature, Editorials, 2023). 

Rethinking Assessment Strategies and Approaches

While the question of whether or not ChatGPT should be used by students and researchers is an emerging and hot topic of discussion, there are currently efforts underway by academic and educational institutions to proactively develop recommended practices for alternative assessment approaches and strategies and academic integrity policies and regulations. These efforts aim to ensure academic integrity through the critical examination of the ways in which assessment should be conducted to prevent k-12, undergraduate, and graduate students from heavy reliance on AI tools such as ChatGPT. A number of Australian universities are rethinking how they can create and run exams and other assessments, arguing for more pen-and-paper exams and tests (Cassidy, 2023). Similar approaches are taken by some universities in the UK. For example, a computer science department in the UK is removing the option of essay-based assignment and solely basing the assessment on skills-based assessment methods. 

Other methods of ensuring academic integrity may include proctoring software to monitor students’ screens and prevent them from accessing and using any unauthorized tools or resources during the exam. Specific examples may include secure browsers and other monitoring tools that prevent students from accessing external websites, resources, or applications during the exam process. Some researchers have recommended embedding images in exam questions and requiring students to demonstrate their knowledge and learning through oral, real-time, and in-person exams (Susnjak, 2022). In addition, it is recommended that specific instructions be given to students prior to exams and tests to encourage them to be ethical and honest and to avoid any AI-enabled tools such as ChatGPT.

Are There Applications to Detect ChatGPT-Generated Texts?

There are a number of applications such as Grammarly, Turnitin, and Copyscape that can detect text similarities. However, these tools have limitations as they rely on their own underlying text databases, which are limited in coverage. This area of research and development is emerging and evolving. One of the latest developments related specifically to ChatGPT is an application called Chatzero (Humans Deserve The Truth) developed by Edward Tian, a Canadian Student in the Princeton Computer Science program that measures the perplexity of input text to detect whether or not a text has been generated by an AI agent or human and provides a plagiarism score. While this application is at its early stage of development and maturity, given my few initial tests on ChatGPT-generated texts, it already shows promise. There is no doubt that the debate over whether or not we should allow students to use AI-enabled tools such as ChatGPT in academia will continue for years. Information science educators and researchers have the knowledge and opportunity to make critical and proactive contributions to the ways in which digital and data literacy and information search and retrieval behaviour research can inform the development and responsible use of AI tools such as ChatGBT. 

References 

Cassidy, C. (2023, January 10). Australian universities to return to ‘pen and paper’ exams after students caught using AI to write essays. The Guardian. https://www.theguardian.com/australia-news/2023/jan/10/universities-to-return-to-pen-and-paper-exams-after-students-caught-using-ai-to-write-essays 

Friesen, J. (2023, January 30). ChatGPT prompting some professors to rethink how they grade students. The Globe and Mail. https://www.theglobeandmail.com/canada/article-chatgpt-ai-post-secondary-students/ 

Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv, 2022-12.

Kelly, S. M. (2023, January 26). ChatGPT passes exams from law and business schools. CNN Business 

https://www.cnn.com/2023/01/26/tech/chatgpt-passes-exams/index.html

Nature. Editorials. Volume 613, January 26, 2023.

Shah, C. (2022, December 20). The rise of AI chat agents and the discourse with dilettantes. Information Matters, Vol. 2, Issue 12. https://informationmatters.org/2022/12/the-rise-of-ai-chat-agents-and-the-discourse-with-dilettantes/ 

Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity? arXiv preprint arXiv:2212.09292.

Weale , S. (2023, January 13). Lecturers urged to review assessments in UK amid concerns over new AI tool. The Guardian. https://www.theguardian.com/technology/2023/jan/13/end-of-the-essay-uk-lecturers-assessments-chatgpt-concerns-ai 

Yang, M. (2023, January 6). New York City schools ban AI chatbot that writes essays and answers prompts. The Guardian. https://www.theguardian.com/us-news/2023/jan/06/new-york-city-schools-ban-ai-chatbot-chatgp